Search for: All records

Inspired by the success of Self-Supervised Learning (SSL) in learning visual representations from unlabeled data, a few recent works have studied SSL in the context of Continual Learning (CL), where multiple tasks are learned sequentially, giving rise to a new paradigm, namely Self-Supervised Continual Learning (SSCL). It has been shown that the SSCL outperforms Supervised Continual Learning (SCL) as the learned representations are more informative and robust to catastrophic forgetting. However, building upon the training process of SSL, prior SSCL studies involve training all the parameters for each task, resulting to prohibitively high training cost. In this work, we first analyze the training time and memory consumption and reveals that the backward gradient calculation is the bottleneck. Moreover, by investigating the task correlations in SSCL, we further discover an interesting phenomenon that, with the SSL-learned background model, the intermediate features are highly correlated between tasks. Based on these new finding, we propose a new SSCL method with layer-wise freezing which progressively freezes partial layers with the highest correlation ratios for each task to improve training computation efficiency and memory efficiency. Extensive experiments across multiple datasets are performed, where our proposed method shows superior performance against the SoTA SSCL methods under various SSL frameworks. For example, compared to LUMP, our method achieves 1.18x, 1.15x, and 1.2x GPU training time reduction, 1.65x, 1.61x, and 1.6x memory reduction, 1.46x, 1.44x, and 1.46x backward FLOPs reduction, and 1.31%/1.98%/1.21% forgetting reduction without accuracy degradation on three datasets, respectively. 

Abstract Spillover of adsorbed species from one active site to another is a key step in heterogeneous catalysis. However, the factors controlling this step, particularly the spillover of polyatomic species, have rarely been studied. Herein, we investigate the spillover dynamics of H* and CH₃* species on a single‐atom alloy surface (Rh/Cu(111)) upon the dissociative chemisorption of methane (CH₄), using molecular dynamics that considers both surface phonons and electron‐hole pairs. These dynamical calculations are made possible by a high‐dimensional potential energy surface machine learned from density functional theory data. Our results provide compelling evidence that the H* and CH₃* can spill over on the metal surface at experimental temperatures and reveal novel dynamical features involving an internal motion during diffusion for CH₃*. Increasing surface temperature has a minor effect on promoting spillover, as geminate recombinative desorption becomes more prevalent. However, the poisoning of the active site can be mitigated by the frequent gaseous molecular collisions that occur under ambient pressure in real‐world catalysis, which transfer energy to the trapped adsorbates. Interestingly, the bulky CH₃* exhibits a significant spillover advantage over the light H* due to its larger size, which facilitates energy acquisition. These insights help to advance our understanding of spillover in heterogeneous catalysis. 

Methane (CH4) oxidation is an important reaction to reduce the greenhouse effect caused by incomplete combustion of CH4. Here, we explored the mechanism of CH4 oxidation catalyzed by CeO2 and Ni-doped CeO2, focusing on the redox properties of these catalyst surfaces, using density functional theory (DFT). We found that the barriers for CH4* activation and H2O* formation are correlated with the surface redox capacity, which is enhanced by Ni doping. Furthermore, the complete reaction mechanism is explored by DFT calculations and microkinetic simulations on bare and Ni-doped CeO2 surfaces. Our calculations suggest that the doping of Ni leads to a much higher overall reactivity, due to a balance between the CH4* activation and H2O* formation steps. These results provide insights into the CH4 oxidation mechanism and the intrinsic relationship between redox properties and the activity of CeO2 surfaces.